1. Field of the Invention
The present invention relates to a method for predicting and encoding a digital image on the assumption that an image signal is transmitted through the effect of a transmitting device provided with various kinds of transfer rates such as an analog phone line, a digital phone line, and a leased data transmission line and is recorded on a storage medium having various kinds of storage capacities such as a magnetic disk or a RAM.
2. Description of the Related Art
In a case that a plurality of image signals corresponding to spatial resolutions are encoded into a single bit stream, as shown in FIG. 1, the use of only one bit stream advantageously makes it possible to correspond to the display units which have their own spatial resolutions only if the bit stream represents the same image.
In order to more efficiently encode the single bit stream than individually encode images having the respective resolutions, the prediction based on a low-resolution image is used for encoding a high-resolution image. As shown in FIG. 2, the encoding based on the prediction takes the steps of encoding a low-resolution image, predicting a high-resolution image from the low-resolution image, and encoding a difference between the predicted image and the high-resolution image. As shown in FIG. 3, the decoding side decodes only the part of the bit stream for composing the low-resolution image or all of the bit stream for composing the high-resolution image.
The process shown in FIG. 2 is implemented by the encoding unit arranged as shown in FIG. 4. The encoding unit shown in FIG. 4 is inputted with the low-resolution image at a low-resolution image input terminal 11 or with the high-resolution image at a high-resolution image input terminal 18. Those inputted images are hierarchically encoded and then is outputted from a high-resolution image output terminal 17.
The image inputted at the low-resolution image input terminal 11 is encoded by a low-resolution image encoder 12 for generating the corresponding low-resolution image codes. The low-resolution image codes are locationally decoded. Then, a high-resolution image predictor 13 operates to predict the high-resolution image from the locally decoded image. The high-resolution image is sent to a difference calculator 14. The difference calculator 14 operates to calculate a difference between the predicted high-resolution image and the high-resolution image inputted at the high-resolution image input terminal 18. The difference is outputted as a difference image. The difference image codes generated by encoding the difference image through the difference image encoder 15 is multiplexed with the low-resolution image codes generated by the low-resolution image encoder 12 through a multiplexer 16. Then, the multiplexed codes are outputted from the high-resolution image output terminal 17.
The process shown in FIG. 12 is implemented by a decoding unit arranged as shown in FIG. 5. The decoding unit shown in FIG. 5 operates to decode the high-resolution image hierarchical codes to be inputted at a high-resolution image code input terminal 21 and then output the low-resolution image at a low-resolution image output terminal 28 and the high-resolution image at a high-resolution image output terminal 27.
The high-resolution image hierarchical codes inputted at the high-resolution image code input terminal 21 is de-multiplexed by a de-multiplexer 22 so that those codes are separated into the low-resolution image codes and the difference image codes. The low-resolution image codes are decoded by a decoder 23. If a user wants to watch the low-resolution image, the decoded image is outputted at a low-resolution image output terminal 28. If a user wants to watch the high-resolution image, the low-resolution image codes are sent to a high-resolution image predictor 24. The predictor 24 operates to predict the high-resolution image from the low-resolution image. The predicted image is sent to an adder 26. The adder 26 operates to add the predicted image to the difference image decoded by the difference image codes for composing the high-resolution image. The high-resolution image is outputted at the high-resolution image output terminal 27.
In general, the high-resolution image predictor 13 provided in the encoding unit shown in FIG. 4 and the high-resolution image predictor 24 provided in the decoding unit shown in FIG. 5 use a linear interpolating filter for doing their predictions.
As an example, the description will be oriented to the predicting method executed in the case of compressing a low-resolution image and a high-resolution image whose resolution is twice as numerous as the low-resolution image in width and length in combination. This predicting method is regulated by spatial scalability of the MPEG2 that is a method for compressing a moving picture.
Assuming that the horizontal number and the vertical number of pixels of the low-resolution image are denoted by H and W, respectively, therefore, the horizontal number and the vertical number of pixels of the high-resolution image are decoded by 2W and 2H, respectively.
Each pixel of the low-resolution image is assumed as B(x, y) x={0, 1, . . . , W-1}, and y={0, 1, . . . , H-1}, while each pixel of the high-resolution image is assumed as U(x, y) x={0, 1, . . . , 2W-1}, and y={0, 1, . . . , 2H-1}.
The linear interpolating filter for predicting the high-resolution image from the low-resolution image is represented by the following expression 1. The phase relation of the pixels between the low-resolution image and the high-resolution image is represented by FIG. 6. ##EQU1## where // represents an operation of dividing a number and rounding it off to an integer.
In case no pixel exists outside an effective image area, such as B(-1, 0), the nearest effective pixel value thereto is taken in place. That is, ##EQU2##
The foregoing process shown in FIG. 6 is implemented by a predictor arranged as shown in FIG. 7. That is, the high-resolution image predictor 13 provided in the encoding unit shown in FIG. 4 and the high-resolution image predictor 24 provided in the decoding unit shown in FIG. 5 are realized by the predictor shown in FIG. 7, respectively.
The predictor shown in FIG. 7 includes a plurality of delaying elements 32, 33, 34 whose delaying times are 1, W-1, and 1, respectively. The low-resolution image is applied from a low-resolution image input terminal 31 into selectors 35 to 38 through the delaying elements 32, 33 and 34. The selectors 35 to 38 are basically operated to directly output the pixel values sent from the delaying elements 32, 33 and 34. The terminals with "0" are used when the pixel value is located outside of an effective pixel range. The pixel values are weighted by a group of multiplexers 39 and then are added by adders 40A, 40B, 40C, and 40D. Then, those values are normalized by dividers 41A, 41B, 41C, and 41D and then are applied to the terminals of a selector 42. Since the outputted image have a double resolution rather than the inputted image in width and length, the number of the pixels of the outputted image is four times as great as that of the inputted image. Hence, assuming that the low-resolution image has a frequency of fs, the high-resolution image has a frequency of 4 fs. Each time one pixel of the image is inputted, the selector 42 operates to sequentially select four terminals and output the image at four times as high a frequency as the inputted image from a predicted high-resolution image output terminal 43.
The predicted high-resolution image is sequentially outputted every pair of lines. Hence, after the predicted high-resolution image is outputted from a predicted high-resolution image output terminal 43, those pairs of lines are rearranged by a sequence converter (not shown).
The foregoing description has concerned with the case that the high-resolution image has twice as large a resolution as the low-resolution image in width and length. In general, the vertical or horizontal magnification may be any different value from each other. When the high-resolution pixel U to be predicted is located at an internally dividing point between the low-resolution pixels B1 and B2 at a ratio of a:(1-a), the value of the pixel U is predicted by the following expression 2. The phase relation of the pixels between the low-resolution image and the high-resolution image is represented by FIG. 8. EQU U=(1-a).times.B1+a.times.B2 expression 2
The vertically and the horizontally one-dimensional interpolated results, both of which are independent of each other, are the pixel values of the predicted high-resolution image.
Further, one of the images to be encoded by this method is a key signal. The key signal represents a shape of an object by its interior or exterior. The object contained in the image is cut out along the shape and is synthesized with another background image. The image cut out by the key signal is termed a fil (see FIG. 9).
The key signal is categorized into a hard key and a soft key. The hard key is an binary image, which digitally represents the interior or the exterior of an object (with 1 and 0, for example). The soft key is a multi-valued (8-bit, for example) image, which represents the interior or the exterior of an object with a maximum value (255) or a minimum value (0) and represents a middle value between the pixel values of the object and the background with a medium value.
The foregoing method for predicting the high-resolution image from the low-resolution image is executed for the normal multi-valued image each pixel of which is represented by 8 bits or 10 bits. This predicting method has the following drawback.
That is, the calculation of each pixel of the high-resolution image needs as many as four pixel values. It means that this calculation is so complicated. Moreover, the calculation needs multiplications and additions. Further, when calculating a peripheral portion of the image, the exceptional process is required for avoiding the reference of the pixels outside of the effective image range.
In particular, in the case of applying this method to the soft key signal, an error takes place in lots of pixels because of the characteristics of the expression 1 served as a lowpass filter.
The present invention is made by overcoming the drawback and it is an object of the present invention to provide a method for predicting a high-resolution image based on a low-resolution image through a minimum process.